BorrowerAPR

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
## 0.00653 0.15629 0.20976 0.21883 0.28381 0.51229      25

The most frequent BorrowerAPR should still be around 0.2. Another peak is around 0.36

Then we want to explore each variable one by one

CreditGrade

##          AA     A     B     C     D     E  NA's 
## 85125  3509  3315  4389  5649  5153  3289  3508

What’s the meaning of ‘NC’? not correct? remove ‘NC’. C Credit Grade is with big probabiltiy, lots of loan records have no CreditGrade information, this is reasonable, because CreditGrade is used for assessing the loan before 2009 July. After 2009 July, we will use ProsperRating for each loan. As we know, CreditGrade or ProsperRating should be one import factor that impacts the APR. There’s no “HR” level in CreditGrade, is ‘NA’ ‘HR’ level?

Something wrong here, NA value should be ‘HR’, change NA to HR.

##          AA     A     B     C     D     E    HR 
## 85125  3509  3315  4389  5649  5153  3289  3508

Term

##    12    36    60 
##  1614 87778 24545

Doubt that may Term 12 has been canceled in the latest prosper loan, however, from the data, the creation time is not old, so this thought is wrong.

There are just three values for Term variable, the most frequent one is 36, three years.

LoanStatus

##              Cancelled             Chargedoff              Completed 
##                      5                  11992                  38074 
##                Current              Defaulted FinalPaymentInProgress 
##                  56576                   5018                    205 
##   Past Due (>120 days)   Past Due (1-15 days)  Past Due (16-30 days) 
##                     16                    806                    265 
##  Past Due (31-60 days)  Past Due (61-90 days) Past Due (91-120 days) 
##                    363                    313                    304

This LoanStatus is not the feature we care about for BorrowerAPR prediction, however, this one may can be used for predicting what kind of loan will be charged-off. This status WOW me, the probabiltiy for defaulted and charged-off is not small.

BorrowerRate

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.0000  0.1340  0.1840  0.1928  0.2500  0.4975

This feature is highly related with BorrowerAPR, BorrowerAPR = BorrowerRate + OrganizationFee. Will check whether organizationFee changes with Credit Grade or not, from the introduction is Prosper company, seems yes.

ProsperRating..numeric.

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   1.000   3.000   4.000   4.072   5.000   7.000   29084

Numerice and alpha value describe the same thing, so can just keep one

ProsperRating..Alpha.

##          AA     A     B     C     D     E  NA's 
## 29084  5372 14551 15581 18345 14274  9795  6935

Something wrong here, NA value should be ‘HR’, change NA to HR.

##          AA     A     B     C     D     E    HR 
## 29084  5372 14551 15581 18345 14274  9795  6935

As we talked before, combine CreditGrade and ProsperRating two columns to one column CreditRating that can describe the credit value.

Create CreditRating

##           A    AA     B     C     D     E    HR 
##   272 17866  8881 19970 23994 19427 13084 10443

Still have 131 loans that are with no CreditRating information.

##          AA     A     B     C     D     E    HR 
##   272  8881 17866 19970 23994 19427 13084 10443

All the credit information is combined. best -> worst, ‘AA’ -> ‘HR’.

ProsperScore

##     1     2     3     4     5     6     7     8     9    10    11  NA's 
##   992  5766  7642 12595  9813 12278 10597 12053  6911  4750  1456 29084

why there’s 11? in data decription file, 10 should be the highest value. What ever, best -> worst, 11 -> 1.

ListingCategory..numeric.

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   0.000   1.000   1.000   2.774   3.000  20.000

The biggest probability is used for Debt consolication.

BorrowerState

Did not understand the meaning of the two letter abbreviation.

Occupation

The occupation should not be one key feature. Most borrowers choose the professional.

EmploymentStatus

##                    Employed     Full-time Not available  Not employed 
##          2255         67322         26355          5347           835 
##         Other     Part-time       Retired Self-employed 
##          3806          1088           795          6134

The classes of EmploymentStatus are weired, employed, not employed, what’s the relationshipe between employed and Full-time, Part-time. Want to combine the levels to just two, employed and not employed.

Create EmploymentFlag

##     Employed Not employed 
##       113102          835

Most borrowers are employed.

EmploymentStatusDuration

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##    0.00   26.00   67.00   96.07  137.00  755.00    7625

This is one long tail variable.

IsBorrowerHomeowner

## False  True 
## 56459 57478

False and True are nearly 50% and 50%.

CreditScoreRangeLower and Upper should be combined into one range column, like income range.

Create CreditScoreRange

Most Borrowers credit score in range 640 - 740. The uppper value = lower value + 19, so we can just keep one for the next revision.

Try to build a new feature to reduce the CreditScoreRange level so check whether will improve the relationship

Create CreditScoreRevision

Add one variable to judge the length of credit history, the longer history, should the lower APR.

Create LengthHistory

The unit is day.

OpenRevolvingAccounts

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    0.00    4.00    6.00    6.97    9.00   51.00

The most frequent count is around 5.

InquiriesLast6Months and TotalInquiries

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.000   1.000   1.435   2.000 105.000     697

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   2.000   4.000   5.584   7.000 379.000    1159

Most InquiriesLast6Months are smaller than 2, outliers can be up to around 100. Most TotalInquiries are smaller than 10, some of outliers can be up to around 300.

CurrentDelinquencies and AmountDelinquent

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.0000  0.0000  0.0000  0.5921  0.0000 83.0000     697

##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max.     NA's 
##      0.0      0.0      0.0    984.5      0.0 463881.0     7622

75% CurrentDelinquencies is 0, most of CurrentDelinquence is smaller than 10, the outlier can be up to 80. The most frequent amountDelinquent not 0 is around 1000.

DelinquenciesLast7Years and TradesNeverDelinquent..percentage.

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.000   0.000   4.155   3.000  99.000     990

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.820   0.940   0.886   1.000   1.000    7544

The variance of TradesNeverDelinquent..percentage. is not so small, except the 25% borrowers, the other ones all have ever missed the repayment time. However, for DelinquenciesLast7Years, 50% borrowers have 0 delinquence.

PublicRecordsLast12Months and PublicRecordsLast10Years

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.000   0.000   0.015   0.000  20.000    7604

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.0000  0.0000  0.0000  0.3126  0.0000 38.0000     697

For this type of variable, actually, want to check, if ignore 0 value, will increase the relationshipe between it and BorrowerAPR?

RevolvingCreditBalance

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##       0    3121    8549   17599   19521 1435667    7604

The most frequent value is around 1e+04

BankcardUtilization and AvailableBankcardCredit

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.310   0.600   0.561   0.840   5.950    7604

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##       0     880    4100   11210   13180  646285    7544

Most BankcardUtilization is smaller than 1, the most frequent AvailableBankcardCredit is around 10000.

DebtToIncomeRatio

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   0.000   0.140   0.220   0.276   0.320  10.010    8554

The most frequent one is around 0.2.

IncomeRange

What’s the meaning of ‘not displayed’, what’s the difference between ‘not employed’ and ‘$0’?

IncomeVerifiable

##  False   True 
##   8669 105268

Most borrowers are IncomeVerifiable

Save the revision data

What is the structure of your dataset?

There are 113937 obs. of 81 variables in this dataset. Each observation is one loan record. The dataset is collected from Prosper webBank, who is America’s frist peer-to-peer lending marketplace. Borrowers request personal loans and inverstor fund. Knowing more background will make us understand the data more easily and clearly.

What is/are the main feature(s) of interest in your dataset?

Since this is one loan dataset, we care most is the BorrowerAPR. We want to known what factors will impact the BorrowerAPR and build one prediction model. The basic one should be CreditRating, however, some combination of other variables should also be used to build the prediction model.

What other features in the dataset do you think will help support your

investigation into your feature(s) of interest?

In my view, the features may imapct the BorrowerAPR includes at least: ProsperScore, EmploymentStatus, EmploymentFlag, IsBorrowerHomeowener, CreditScoreRange, OpenRevolvingAccounts, InquireisLast6Months, CurrentDelinquencies, AmountDelinquent, DelinquenciesLast7Years, PublicRecordsLast10Years, PublicRecordsLast12Months, BankcardUtilization, AvailableBankcardCredit, TradesNeverDelinquent, DebtToIncomeRatio, IncomeRange, IncomeVerifiable, LengthHistory etc.

After lots of research, find that the five important components for credit score, i.e., payment history, credit utilization, length of credit history, new credit and credit mix. The features in data set almost all have relationship with the five components. However, except CreditRating, ProsperScore, CreditScoreRange, which already take acount of the whole history information, Delinquency part, PublicRecords part, BankcardUtilization, AvailableBankcardCredit, DebtToIncomeRatio, LengthHistory should be the most related features.

Did you create any new variables from existing variables in the dataset?

Since CreditGrade is used for period before July, 2009 and ProsperRating is used fo period after July, 2009, i combine them to one CreditRating feature.

Create a variable EmploymentFlag to classify the emopolyment status to employed and not employed, will check more in the Bivariate Plots Section.

Create a variable CreditScoreRange to describe CrediteScoreRangeLower and Upper more directly. And then create a variable CreditRevision to reduce the levels.

Create a variable LengthHistory to judge the length of credit history.

Of the features you investigated, were there any unusual distributions?

Did you perform any operations on the data to tidy, adjust, or change the form
of the data? If so, why did you do this?

GreditGrade and ProsperRating miss ‘HR’ level, change the NA value to ‘HR’, since ProsperRating(numeric) use 1 for ‘HR’.

Change Term, ProsperScore to factor, since just limited levels.

Change FirstRecordedCreditLine and ListingCreationDate to Datetime

Orgination fee of CreditRating

One idea bingo, want to know whether the orgination fee changes with CreditRating, so we will build one variable OrginationFee and visulize it.

We can see that the orgination fee also changes with the CreditRationg level. The obvious diff is between level ‘AA’ and level “A”.

BorrowerAPR of CreditRating

The result meets our expectation as mentioned before, However, why there are lots of outliers and the variance is not small? it seems there are still other variables control the BorrowerAPR trendency, their influence can not be ignored.

Keep CreditRating fixed, chech the influence of ProsperScore.

BorrowerAPR of ProsperScore

As the correlation value calculated before, strong relationship between BorrowerAPR and ProsperScore.

Plot the relationship between ProsperScore and CreditRating. Confused, how to get the CreditRating? how to calculate the ProsperScore?

CreditRating vs ProsperScore

From the figure, we can see, part of ‘AA’ borrowers still have high risk score, e.g., 4. Therefore, we still should keep the ProsperScore feature, it can descirbe a different dimension for BorrowerAPR prediction.

BorrowerAPR of CreditScoreRange

The sample number in both tail side is not enough, e.g., [0 - 440]. The score is got from customer credit rating agency, still that quesion, how to get credit rating?

CreditRating vs CreditScoreRange

From the figure, we can see CreditRating ‘AA’ borrowers may have a low CreditScore, why? Whatever, CreditScoreRange is still one important feature for the prediction.

Try to reduce the levels for CreditScoreRange to check whether can imporve the correlation value by this way.

BorrowerAPR of CreditScoreRevision

##   (0,640] (640,680] (680,720] (720,760] (760,800] (800,840] (840,880] 
##     26605     32858     28394     15873      7268      1976       239 
##      NA's 
##       724

Emmm, more clear than CreditScoreRange.

BorrowerAPR of BankcardUtilization

If understand correctly, this BankcardUtilization should mean ratio of your credit card balances to credit limits. The higher the BankcardUtilization, the higher BorrowerAPR, because high BankcardUtilization will make lender to think that there’s an increased risk.

BorrowerAPR of BankcardUtilization

Bases on before correlation calculation output, we know AvailableBankcardCredit should have weak relationship with BorrowerAPR, this figure shows this. With present knowledge, credit limits = credit balance + credit pending transaction + avaliable credit. This feature can show the credit limits inderectly.

BorrowerAPR of TradesNeverDelinquent..percentage.

The higher radesNeverDelinquent..percentage., the lower BorrowerAPR.

BorrowerAPR of EmploymentStatus and EmploymentFlag

There’s no clear trend in EmploymentStatus figure, however, we can see the BorrowerAPR of ‘Not employed’ is obviously larger than other classes. So, create a new variable EmploymentFlag to check more.

This should be clear now, Employed may get a low Borrower APR, but why the correlation value is low between BorrowerAPR and EmployedFlag? If we check more carefully, the tail below 25% seems long.

BorrowerAPR of CurrentDelinquencies and AmountDelinquent

The Trend is not that obvious, just a little move upward. However, for CurrentDelinquencies, the diff betweet 0 and 1 is much bigger than the diff between 1 and 2, 2 and 3.

BorrowerAPR of DelinquenciesLast7Years

The trend is a little more obvious than CurrentDelinquencies and AmountDelinquent.

BorrowerAPR of IsBorrowerHomeowner

The trend is not that obvious.

BorrowerAPR of InquiriesLast6Months

The Trend is not that obvious, just a little move upward.

BorrowerAPR of TotalTrades

The Trend is not that obvious.

BorrowerAPR of DebtToIncomeRatio

As what

BorrowerAPR of OpenRevolvingAccounts

BorrowerAPR of IncomeRange

0 not reasonable, samples are not enough. What’s the difference between ‘$0’ and ‘not employed’. The trend move a little downward.

BorrowerAPR of IncomeVerifiable

BorrowerAPR of PublicRecordsLast10Years

The same with CurrentDelinquencies, the diff is much larger between 0 and 1. For this kind of variable, acturally, we can build a new variable, 0 and not 0, to increase the relationship, am i right?

IncomeVerifiable vs DebtToIncomeRatio

50% DebtToincome is 10.1 when IncomeVerifiable is false, this is the max value for DebtToincome, what’s the meaning of 10.1? it seems that the strong relationship between DebtToIncomRatio and IncomeVerifiable is meanless and not useful for our objective.

Talk about some of the relationships you observed in this part of the

investigation. How did the feature(s) of interest vary with other features in
the dataset?

There’s very strong relationship between variable BorrowerAPR and CreditRating, this meets our expectation, the higher CreditRating is, the lower BorrowerAPR should be.

Moreover, there’s strong relationship between variable BorrowerAPR and ProsperScore while meanwhile CreditRating also has strong relationship with ProsperScore. Moreover, from the bar plot, part of ‘AA’ borrowers still have high risk score, e.g., 4. Therefore, we still should keep the ProsperScore feature, it can descirbe a different dimension for BorrowerAPR prediction.

BorrowerAPR has moderate relationship with CreditScoreRange while CreditRating also has moderate relationship with CreditScoreRange. Moreover, we can see CreditRating ‘AA’ borrowers may have a low CreditScore, why? Whatever, CreditScoreRange is still one feature for the prediction.

BorrowerAPR has weak relationship with BankcardUtilization, AvaliableBankcardCredit and TradesNeverDelinquent. Meanwhile, they have weak relationship between each other. Moreover, they have moderate relationship with CreditScoreRange.

What confuses me is that, where we get the CreditRating, ProsperScore? All these features should combine the important credit information, like, payment history, credit utilization, length of creit history, new credit, credit mix etc, all of them are history feature.

Did you observe any interesting relationships between the other features

(not the main feature(s) of interest)?

InquiriesLast6Months has strong relationship with TotalInquiries, reasonable. TradesNeverDelinquent has moderate relationship with CurrentDelinquencies, DelinquenciesLast7Years, which makes sense. All these prove that the history can predict the present status.

IncomeVerifiable has strong relationship with DebtToIncomeRatio, then find that when IncomeVeriiable is False, 50% DebtToIncomeRatio is max value 10.1, do not know the meaning of this value, this relathionship should be not useful for our objective.

What was the strongest relationship you found?

BorrowerAPR and CreditRating

BorrowerAPR vs CreditRating vs ProsperScore

The trend is not that obvious, we can still see the small downward with Prosper. However, seems not for D and E.

BorrowerAPR vs CreditRating vs CreditScoreRevision

BorrowerAPR vs CreditRating vs BankcardUtilization

We know there’s weak relationship between CreditRating and BankcardUtilization, the figure proves that, the most obvious one is that ‘AA’ borrowers tend to have small BankcardUtilization, ‘HR’ and ‘E’ borrowers tend to have big BankcardUtilization.

BorrowerAPR vs CreditRating vs TradesNeverDelinquent..percentage.

From the figure, we can see the weak relationship between TradesNeverDelinquent..percentage. and CreditRating, more ‘HR’ borrowers have small TradesNeverDelinquent..percentage. compared with ‘AA’ borrower. Moreover, we can see that if keep the creditRating fixed, smaller TradesNeverDelinquent..percentage tends to have bigger BorrowerAPR.

Talk about some of the relationships you observed in this part of the

investigation. Were there features that strengthened each other in terms of
looking at your feature(s) of interest?

BorrowerAPR has very strong relationship with CreditRating, strong relationship with ProsperScore, moderate relationship with CreditScoreRevision, weak relationship with BankcardUtilization, AvaliableBankcardCredit and TradesNeverDelinquent. Moreover, all these features have strong, moderate, weak relationship with CreditRating. However, if we dig deeper, can find each of them can describe a different dimension view of data. Where, wonder how to get the CreditRating value? calculated with history data, which includes payment history, credit utilization, length of creit history, new credit, credit mix etc? how to get the ProsperScore? how to get CreditScoreRange? what’s their difference?

Were there any interesting or surprising interactions between features?

lots of features have strong and moderate relationship with CreditRating. If all these features, i.e., CreditRating, ProsperScore, CreditScoreRevision, take account of history data, why they are different?, why we need them all?

OPTIONAL: Did you create any models with your dataset? Discuss the

strengths and limitations of your model.

Want to build a model to predict the BorrowerAPR, will do this in future, can use Machine Learning Algrithms.

Plot One

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
## 0.00653 0.15629 0.20976 0.21883 0.28381 0.51229      25

Description One

From this figure, we can see several peaks, the most frequent one is around 0.2 and another bigger peak is around 0.36

Plot Two

Description Two

The higher CreditRating is, the lower BorrowerAPR will be. However, the variance for each CreditRating level is not small, There are still some other imported features to control the finnal BorrowerAPR. From the trendency, it seems that one linear model can be built to predict the BorrowerAPR.

Plot Three

Description Three

From the figure, we can see the weak relationship between TradesNeverDelinquent..percentage. and CreditRating, more ‘HR’ borrowers have small TradesNeverDelinquent..percentage. compared with ‘AA’ borrower. Moreover, we can see that if keep the creditRating fixed, smaller TradesNeverDelinquent..percentage tends to have bigger BorrowerAPR.